Holographic image display system

ABSTRACT

This invention relates to holographic image display systems, and to related methods and processor control code. We describe a method of displaying an image holographically, the method including: inputting display image data defining said image for display; processing said image data to determine first image data representing a first spatial frequency portion of said image data and second image data representing a second spatial frequency portion of said image data, wherein said second spatial frequency is higher than said first spatial frequency; displaying a hologram of said first image data on a spatial light modulator (SLM) to form a holographically-generated intermediate real image; modulating said intermediate real image using said second image data to display said image.

CLAIM OF PRIORITY

This application is a continuation of U.S. patent application Ser. No.12/182,095, filed Jul. 29, 2008, titled HOLOGRAPHIC IMAGE DISPLAYSYSTEM, which claims priority under 35 U.S.C. §119 to United KingdomApplication No. 0813009.8, filed Jul. 16, 2008, each of which isincorporated in its entirety by reference herein.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to holographic image display systems, and torelated methods and processor control code.

2. Description of the Related Art

We have previously described techniques for displaying an imageholographically—see, for example, WO 2005/059660 (Noise SuppressionUsing One Step Phase Retrieval), WO 2006/134398 (Hardware for OSPR), WO2007/031797 (Adaptive Noise Cancellation Techniques), WO 2007/110668(Lens Encoding), and WO 2007/141567 (Colour Image Display). These areall hereby incorporated by reference in their entirety.

Advantages of holographic image display techniques include a compactoptical system and increased optical efficiency as compared withconventional display systems. However the techniques we have previouslydescribed are relatively computationally expensive when displayinghigh-resolution images and, in a hardware (ASIC) implementation theresolution is closely tied to the hardware configuration. In anOSPR-type approach, where multiple spatially-coincident temporalholographic subframes are displayed for each image frame, the powerconsumption may be reduced by employing multi-phase rather than binaryphase modulation, but nonetheless further power savings are desirable.Depending upon the implementation of the technique, more SLM (SpatialLight Modulator) pixels may be employed than are seen as image pixels,which can increase cost and make miniaturisation difficult. A highcontrast image display can also be difficult to achieve.

It is therefore desirable to be able to increase the efficiency andimage quality of holographic image display systems, and to furtherreduce their size.

SUMMARY OF THE INVENTION

According to a first aspect of the present invention there is thereforeprovided a method of displaying an image holographically, the methodcomprising: inputting display image data defining said image fordisplay; processing said image data to determine first image datarepresenting a first spatial frequency portion of said image data andsecond image data representing a second spatial frequency portion ofsaid image data, wherein said second spatial frequency is higher thansaid first spatial frequency; displaying a hologram of said first imagedata on a spatial light modulator (SLM) to form a holographicallygenerated intermediate real image; modulating said intermediate realimage using said second image data to display said image.

In embodiments of the method the resolution of an SLM displaying thehologram may be relatively low (lower than that of an SLM intensitymodulating the intermediate real image), and therefore computation ofthe hologram becomes straightforward, especially when using an OSPR-typeapproach such as ADOSPR (adaptive OSPR). In a hardware implementation asmall, cheap and very low power ASIC (Application Specific IntegratedCircuit) may be employed. Simulations have shown that an SLM resolutionof 64×64, or even lower, may be sufficient for the hologram.

In embodiments of the method the resolution of an SLM displaying thehologram may be selected substantially independently of a desiredresolution of the image. Thus the resolution of the SLM hologram may beselected simply dependent on the typical proportion of image energycontained within its low frequency components. Increasing the resolutionof video images adds energy to high frequency components but does notsubstantially change the low frequency content. Thus the resolution ofan image projection system embodying the method may be changed, forexample increased without modifying the hologram SLM or the associatedhologram data processing. Therefore in embodiments in which the hologramdata processing is performed by dedicated hardware such as an ASIC, theprojector resolution may be increased without modifying, or withoutsubstantially modifying, the ASIC.

Embodiments of the method are able to provide a high contrast displaysince residual background noise, which is of low energy but perceptuallysignificant, is substantially blocked an imaging SLM modulating theintermediate real image using the second image data. In embodiments ofthe method no error diffusion (such as that we have described inprevious patent applications) is employed to reduce background noise,thus further reducing the computational cost.

A holographic image projector embodying the method can be readilyminiaturisable. This is because the SLM displaying the hologram has arelatively small active area due to its low resolution (smaller than theimage resolution) and therefore physically small illumination optics maybe employed. Further there is no particular need for pixels of the SLMdisplaying the hologram to have a very small lateral dimension in orderto shrink the illumination optics. Thus, for example, the illuminationoptics can be very small even with the current generation of SLMs whichfeature for example 5.62 μm pixels.

Since the intermediate real image is intensity modulated, in embodimentsby an intensity modulating SLM of a higher resolution that a phasemodulating SLM displaying a hologram, in embodiments a proportion of thelight of the intermediate real image is blocked by the imaging(intensity modulating) SLM, to form the image. However this is much lessthan the light blocked in a conventional imaging system, and althoughthe system may be slightly less optically efficient than a “pure”holographic projector the overall system efficiency can be significantlyhigher due to the considerably reduced computational requirements.

In embodiments of a holographic image projector embodying the method (asdescribed further below) alignment between an SLM displaying thehologram and an SLM modulating the intermediate real image produced bythe hologram is important. Very broadly speaking, the hologram forms alow resolution intermediate real image which is intensity modulatedusing a higher resolution SLM to add the high frequency spatialcomponents not present in the image formed by the hologram. Ideallytherefore the alignment between the intensity modulating SLM and the SLMdisplaying the hologram should be to within one to two pixels of theintensity modulating SLM—that is the intensity modulating SLM adding thehigh frequency components should have pixels which line up within one totwo pixels with the pixel boundaries of the intermediate real imageformed by displaying the hologram. In practice, however, the mechanicalalignment requirements can be relaxed with the addition of defocusand/or phase gradients to the holograms on the hologram SLM, to move theintermediate real image formed by the hologram SLM axially and laterallyto the correct position on the surface of the intensity modulating SLM.(Techniques for encoding lens power into a hologram, in particular inthe context of encoding one or more lenses for the optics of anOSPR-type holographic image projection system, are described in WO2007/110668; this description is hereby incorporated by reference).

In embodiments of the method low and high spatial frequency componentsof the image data are extracted to provide the first (low resolution)and second (high resolution) image data. A hologram is generated fromthe low resolution image data such that when the hologram is displayedon the hologram SLM it reproduces a version of the low resolution imagedata comprising the low spatial frequencies of the image. The skilledperson will understand that in embodiments displaying the hologramcomprises displaying multiple, temporal subframes which noise-average togive a version of the low spatial frequency component of the image data,thus providing the intermediate real image. In general the intermediatereal image will not be a precisely accurate reproduction of the lowspatial frequency portion of the image data since it will haveassociated noise. Thus preferred embodiments of the method calculate theexpected intermediate real image (including the noise) and thendetermine the high spatial frequency component of the image data whichis to be displayed on the intensity modulating SLM as that (high spatialfrequency) portion of the image data which is left over from theintermediate real image. The intensity modulation comprises, in effect,a multiplication of the intermediate real image by the pattern on theintensity modulating SLM (the second image data). Thus to determine thehigh spatial frequency components left over from the holographic displayof the lower spatial frequency components, in embodiments the image datais divided by the intermediate real image which is calculated to beformed by the displayed hologram.

Since an intensity modulating SLM only removes light from theintermediate real image (by blocking light) in some preferredembodiments the first image data from which the hologram displayed onthe hologram SLM is generated comprises a reduced resolution of theimage data in which each reduced resolution pixel has a value dependenton the image pixels from which it is derived, preferably (but notnecessarily) a peak value of the image pixel values from which it isderived. Using a peak value enables values of pixels of the desiredimage for display to be accurately represented by blocking light, but inother implementations other statistics may additionally or alternativelybe employed. For example in embodiments of the method it may be usefulto modify peak values used for pixels of the first image data from whichthe intermediate real image is holographically generated to compensatefor edge effects. Thus not every image pixel need have a replacementpixel value as described.

As previously mentioned, the intermediate real image will generally havea certain amount of noise. It might be thought that to accuratelyrepresent the second (high spatial frequency) image data one would haveto determine the lowest instantaneous level of a pixel in theintermediate real image and then scale the high spatial frequencyinformation displayed on the intensity modulating SLM accordingly (sincethis block pixel of the intermediate real image would effectively limitthe maximum light output for the corresponding portion of the desiredimage to display). However, depending upon the statistical properties ofthe noise, this could result in a relatively inefficient system if, forexample, the noise resulted in one intermediate image pixel having aparticularly low brightness. For this reason it is preferable to balanceaccurate rendition of the image with optical efficiency and thereforenot necessarily to lower the maximum light output for a region of theimage so that it is below an actually achievable brightness for thatregion of the image. Instead it may be tolerable to allow some pixels ofthe displayed image to have a lower than ideal value if, by doing this,the overall optical efficiency can be increased. One way of achievingthis is to determine a scaling factor for the second (high frequency)image data using a calculated version of the (noisy) intermediate realimage, and then to scale this scaling factor to increase the opticalefficiency by blocking less light overall, at the expense of introducinga slight amount of “clipping”. The skilled person will, however,appreciate that other techniques may be employed to achieve a similarresult.

In some preferred embodiments of the method an OSPR-type technique isemployed to display the hologram—that is the hologram is displayed bydisplaying a plurality of holographic subframes, each when replayedhaving substantially the same spatial coverage, in rapid succession toaverage to the desired result. This is computationally much lessexpensive than other techniques. However it is particularly preferred toemploy an adaptive OSPR-type technique in which the (deliberatelyintroduced) noise in each successive temporal subframe aims to at leastpartially compensate for noise in the replayed image arising from thedisplay of one or more previous temporal subframes. This approach isuseful because it helps to reduce the risk of “spikes” in the noisewhich can have the effect of pushing down the base line level of lightin the displayed image, as described above. A low-level “ripple”-typenoise allows a base line level which is relatively close to the medianavailable light level at the intermediate real image whereas a spikewill tend to depress the base line level, and hence the opticalefficiency of the system and/or introduce noise (if, in effect, it isignored).

In some preferred implementations the image is displayed by projectinglight from the intensity modulating SLM towards a screen. Preferably, toreduce the effect of speckle, the projection optics include a diffuserat a further intermediate real image plane, in particular a planecomprising a real image from the hologram SLM modulated by the intensitymodulating SLM. There is a trade off between speckle reduction and depthof field of the final projected image, depending on the diffusion angleof the diffuser: a greater angle reduces speckle, but also reduces depthof field (a diffusion angle of zero degrees, in effect if the diffuseris absent, results in a projected image which is substantially in focusat substantially all distances from the projector). In some particularlypreferred embodiments the diffuser is mechanically driven, in particularusing a piezoelectric actuator, for example in conjunction with a binaryphase diffuser. Preferably a minimum feature size or pixel pitch of thediffuser is less than a pixel pitch of the further intermediate realimage (at the diffuser). In this way speckle may be reduced at increasedspatial frequencies than would otherwise be the case (with a largerpixel pitch the diffuser can have the effect of adding more OSPR-typesubframes). A synergistic effect has been observed in visual noisereduction in an OSPR-type holographic image display system using aspeckle reducing diffuser with pixels smaller than those of theintermediate image at which the diffuser is located. In embodiments thepiezoelectric actuator may have a stroke of at least 5 μm, morepreferably at least 10 μm (and/or the diffuser is preferably moved bymore than 2, 5 or 10 diffuser pixels within the duration of an imageframe comprising one or more temporal subframes). Further details of theapplication of a diffuser to OSPR-based and other types of holographicimage display systems can be found in our earlier UK patent applicationGB 0800167.9 of 7 Jan. 2008, hereby incorporated by reference.

In preferred embodiments a multicolour, preferably a full colour imagedisplay is provided. This may be achieved by combining light from red,green and blue lasers (for example wavelengths of, broadly speaking,greater than 600 nm, 500-600 nm, and less than 500 nm). These may becombined and provided as a single, colour time-multiplexed beam to thehologram SLM. Since in embodiments the hologram SLM may have arelatively small number of pixels, for example equal to or less than512, 256, 128, 64 or 32 pixels (in the x and/or y-direction) the activearea of the hologram SLM may be relatively small, for example with amaximum lateral dimension of less than 1 mm. This facilitates shrinkingthe optics illuminating the hologram SLM, and hence embodiments of theholographic image display.

In a related aspect the invention provides a method of displaying animage holographically, the method comprising dividing said image intolower and higher resolution representations of said image, displaying ahologram of said lower resolution representation of said image on aphase modulator to generate a lower resolution representation of saidimage, and modulating an intensity of said lower resolutionrepresentation of said image using said higher resolution representationof said to display said image.

As previously mentioned, in some preferred implementations the higherresolution representation of the image is determined such that acombination of the holographically generated lower resolutionrepresentation of the image and the higher resolution representation ofthe image together approximate the desired image for display. Theapproximation need not be precisely accurate since it may be desirabledeliberately to introduce a small amount of noise in order to increasethe overall optical efficiency of the system by “clipping” some of thepixels in the displayed image where noise in the holographicallygenerated image would otherwise dictate an overall reduced displayedimage brightness.

In a further related aspect there is provided a holographic imagedisplay system comprising means to implement aspects and embodiments ofthe above-described methods.

Thus in one aspect such a system comprises means for inputting displayimage data, processing this to determine hologram data for displaying a,low spatial frequency image and intensity modulation data for modulatingthe low spatial frequency image with higher spatial frequency componentsof the desired image for display, in order to display a desired image.

In another related aspect the invention provides a holographic imagedisplay system including means to divide the image into lower and higherresolution representations of the image, and means for displaying ahologram at a lower resolution representation and for modulating anintensity of an image replayed from the displayed hologram (or pluralityof holographic subframes) in order to display the image).

In a still further related aspect the invention provides a system fordisplaying an image holographically, the system comprising: an input toreceive display image data defining said image for display; a processorto process said image data to determine first image data representing afirst spatial frequency portion of said image data and second image datarepresenting a second spatial frequency portion of said image data,wherein said second spatial frequency is higher than said first spatialfrequency; an output to output data for displaying a hologram of saidfirst image data on a spatial light modulator (SLM) to form aholographically-generated intermediate real image; and an output tooutput data for modulating said intermediate real image using saidsecond image data, to thereby display said image.

Embodiments of the above-described system may be implemented in eitherhardware or software or a combination of the two. A common output orseparate output may be employed for driving respective phase andintensity modulating spatial light modulators for displaying thehologram and modulating the intermediate real image.

In a still further related aspect the invention provides a method ofprocessing data for displaying an image holographically the methodcomprising: inputting display image data defining said image fordisplay; processing said image data to determine first image datarepresenting a first spatial frequency portion of said image data andsecond image data representing a second spatial frequency portion ofsaid image data, wherein said second spatial frequency is higher thansaid first spatial frequency; generating data for displaying a hologramof said first image data on a spatial light modulator (SLM) to form aholographically-generated intermediate real image; and generating datafor modulating said intermediate real image using said second image datato thereby display said image.

In embodiments the determining of the second (high spatial frequency)image data comprises calculating a reconstruction of the displayedhologram and then processing the image data using this, for exampledividing by this calculated data to determine the second (higher spatialfrequency) image data, that is the remaining spatial frequency componentof the image data to be added to the holographically generatedintermediate real image to reproduced a desired image.

In a related aspect the invention provides a carrier carrying processorcontrol code for implementing a method as described above.

The carrier may be, for example, a disk, CD- or DVD-ROM, or programmedmemory such as read-only memory (Firmware). The code (and/or data) maycomprise source, object or executable code in a conventional programminglanguage (interpreted or compiled) such as C, or assembly code, forexample for general purpose computer system or a digital signalprocessor (DSP), or the code may comprise code for setting up orcontrolling an ASIC (Application Specific Integrated Circuit) or FPGA(Field Programmable Gate Array), or code for a hardware descriptionlanguage such as Verilog (Trade Mark) or VHDL (Very high speedintegrated circuit Hardware Description Language). As the skilled personwill appreciate such code and/or data may be distributed between aplurality of coupled components in communication with one another.

In a still further aspect the invention provides a holographic imageprojection system comprising: at least one laser light source; a firstspatial light modulator (SLM) to phase modulator light from said atleast one laser light source; intermediate optics to provide anintermediate real image plane at which a real image produced by ahologram on said first SLM is formed; a second SLM located at saidintermediate real image plane to intensity modulate said real image; andoutput optics to project an image of said intensity modulated realimage; and wherein a resolution of said second SLM is greater than aresolution of said first SLM.

In preferred embodiments the output optics are configured to provide asecond intermediate real image plane for a diffuser, as described above.In embodiments the first (phase modulating) SLM has an active area whichis smaller, for example less than half the size of the second (intensitymodulating) SLM. In some preferred embodiments the active area of thefirst SLM has a maximum lateral dimension of less than 1 mm, preferablyless than 0.5 mm. In some preferred embodiments the first SLM is amultiphase SLM (with at least three quantised phase levels) rather thana binary phase SLM for efficiency (this allows suppression of aconjugate holographically generated intermediate real image).

In preferred embodiments the system includes combining optics to combinered, green and blue laser light from time-multiplexed light sources inorder to provide a full colour display. The system may be combined witha controller to provide a holographic image projector as describedabove. In embodiments the controller provides the functions describedabove, and is further configured to control the optical power from theone or more laser light sources, in particular dependent upon thecalculated intermediate real image from the hologram SLM and upon anyscaling applied to increase the optical (diffraction) efficiency of thesystem.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of the invention will now be further described,by way of example only, with reference to the accompanying figures inwhich:

FIG. 1 shows an example image (left) and the corresponding powerspectrum (right);

FIG. 2 shows an embodiment of a holographic image projection systemaccording to the invention;

FIGS. 3 a to 3 d show, respectively, a block diagram of a hologram datacalculation system, operations performed within the hardware block ofthe hologram data calculation system, energy spectra of a sample imagebefore and after multiplication by a random phase matrix, and an exampleof a hologram data calculation system with parallel quantisers for thesimultaneous generation of two sub-frames from real and imaginarycomponents of complex holographic sub-frame data;

FIGS. 4 a and 4 b show, respectively, an outline block diagram of anadaptive OSPR-type system, and details of an example implementation ofthe system;

FIGS. 5 a to 5 c show, respectively, a colour holographic imageprojection system, and image, hologram (SLM) and display screen planesillustrating operation of the system;

FIGS. 6 a and 6 b show, respectively, a procedure, and a system forgenerating N subframe holograms for displaying an enhanced resolutionimage;

FIGS. 7 a to 7 c show, respectively a schematic illustration of theeffect of a diffraction efficiency boost parameter, a softwareimplementation of an embodiment of a dual spatial frequency phase,intensity holographic projection system controller according to anembodiment of the invention, and a hardware implementation of anembodiment of a dual spatial frequency phase, intensity holographicprojection system controller according to an embodiment of theinvention;

FIG. 8 shows (left) a replay field I formed by 16 hologram subframesdisplayed on a phase SLM and (right) the corresponding high-frequencyimage to display on an intensity modulating SLM2 to modulate the field Ito reproduce the Mustang image of FIG. 1; and

FIGS. 9 a to 9 f show examples of the Mustang image for respectivediffraction efficiency boost parameter values, D, of D=1.0, 1.3, 1.5,2.0, 4.0, and 8.0.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Preferred embodiments of the invention use an OSPR-type hologramgeneration procedure, and we therefore describe examples of suchprocedures below. However embodiments of the invention are notrestricted to such a hologram generation procedure and may be employedwith other types of hologram generation procedure including, but notlimited to: a Gerchberg-Saxton procedure (R. W. Gerchberg and W. O.Saxton, “A practical algorithm for the determination of phase from imageand diffraction plane pictures” Optik 35, 237-246 (1972)) or a variantthereof, Direct Binary Search (M. A. Seldowitz, J. P. Allebach and D. W.Sweeney, “Synthesis of digital holograms by direct binary search” Appl.Opt. 26, 2788-2798 (1987)), simulated annealing (see, for example, M. P.Dames, R. J. Dowling, P. McKee, and D. Wood, “Efficient optical elementsto generate intensity weighted spot arrays: design and fabrication,”Appl. Opt. 30, 2685-2691 (1991)), or a POCS (Projection Onto ConstrainedSets) procedure (see, for example, C. -H. Wu, C. -L. Chen, and M. A.Fiddy, “Iterative procedure for improved computer-generated-hologramreconstruction,” Appl. Opt. 32, 5135-(1993)).

OSPR

Broadly speaking in our preferred method the SLM is modulated withholographic data approximating a hologram of the image to be displayed.However this holographic data is chosen in a special way, the displayedimage being made up of a plurality of temporal sub-frames, eachgenerated by modulating the SLM with a respective sub-frame hologram,each of which spatially overlaps in the replay field (in embodimentseach has the spatial extent of the displayed image).

Each sub-frame when viewed individually would appear relatively noisybecause noise is added, for example by phase quantisation by theholographic transform of the image data. However when viewed in rapidsuccession the replay field images average together in the eye of aviewer to give the impression of a low noise image. The noise insuccessive temporal subframes may either be pseudo-random (substantiallyindependent) or the noise in a subframe may be dependent on the noise inone or more earlier subframes, with the aim of at least partiallycancelling this out, or a combination may be employed. Such a system canprovide a visually high quality display even though each sub-frame, wereit to be viewed separately, would appear relatively noisy.

The procedure is a method of generating, for each still or video frameI=I_(xy), sets of N binary-phase holograms h⁽¹⁾ . . . h^((N)). Inembodiments such sets of holograms may form replay fields that exhibitmutually independent additive noise. An example is shown below:

-   1. Let G_(xy) ^((n))=I_(xy) exp(jφ_(xy) ^((n))) where φ_(xy) ^((n))    is uniformly distributed between 0 and 2π for 1≦n≦N/2 and 1≦x, y≦m-   2. Let g_(uv) ^((n))=F⁻¹[G_(xy) ^((n))] where F⁻¹ represents the    two-dimensional inverse Fourier transform operator, for 1≦n≦N/2-   3. Let m_(uv) ^((n))=    {g_(uv) ^((n))} for 1≦n≦N/2-   4. Let m_(uv) ^((n+N/2))=ℑ{g_(uv) ^((n))} for 1≦n≦N/2-   5.

${{Let}\mspace{14mu} h_{uv}^{(n)}} = \{ {{\begin{matrix}{- 1} & {{{if}\mspace{14mu} m_{uv}^{(n)}} < Q^{(n)}} \\1 & {{{if}\mspace{14mu} m_{uv}^{(n)}} \geq Q^{(n)}}\end{matrix}{where}\mspace{14mu} Q^{(n)}} = {{{median}\mspace{14mu}( m_{uv}^{(n)} ){and}\mspace{14mu} 1} \leq n \leq N}} $

Step 1 forms N targets G_(xy) ^((n)) equal to the amplitude of thesupplied intensity target I_(xy), but with independentidentically-distributed (i.i.t.), uniformly-random phase. Step 2computes the N corresponding full complex Fourier transform hologramsg_(uv) ^((n)). Steps 3 and 4 compute the real part and imaginary part ofthe holograms, respectively. Binarisation of each of the real andimaginary parts of the holograms is then performed in step 5:thresholding around the median of m_(uv) ^((n)) ensures equal numbers of−1 and 1 points are present in the holograms, achieving DC balance (bydefinition) and also minimal reconstruction error. The median value ofm_(uv) ^((n)) may be assumed to be zero with minimal effect on perceivedimage quality.

FIG. 3 a, from our WO2006/134398, shows a block diagram of a hologramdata calculation system configured to implement this procedure. Theinput to the system is preferably image data from a source such as acomputer, although other sources are equally applicable. The input datais temporarily stored in one or more input buffer, with control signalsfor this process being supplied from one or more controller units withinthe system. The input (and output) buffers preferably comprise dual-portmemory such that data may be written into the buffer and read out fromthe buffer simultaneously. The control signals comprise timing,initialisation and flow-control information and preferably ensure thatone or more holographic sub-frames are produced and sent to the SLM pervideo frame period.

The output from the input comprises an image frame, labelled I, and thisbecomes the input to a hardware block (although in other embodimentssome or all of the processing may be performed in software). Thehardware block performs a series of operations on each of theaforementioned image frames, I, and for each one produces one or moreholographic sub-frames, h, which are sent to one or more output buffer.The sub-frames are supplied from the output buffer to a display device,such as a SLM, optionally via a driver chip.

FIG. 3 b shows details of the hardware block of FIG. 3 a; this comprisesa set of elements designed to generate one or more holographicsub-frames for each image frame that is supplied to the block.Preferably one image frame, I_(xy), is supplied one or more times pervideo frame period as an input. Each image frame, I_(xy), is then usedto produce one or more holographic sub-frames by means of a set ofoperations comprising one or more of: a phase modulation stage, aspace-frequency transformation stage and a quantisation stage. Inembodiments, a set of N sub-frames, where N is greater than or equal toone, is generated per frame period by means of using either onesequential set of the aforementioned operations, or a several sets ofsuch operations acting in parallel on different sub-frames, or a mixtureof these two approaches.

The purpose of the phase-modulation block is to redistribute the energyof the input frame in the spatial-frequency domain, such thatimprovements in final image quality are obtained after performing lateroperations. FIG. 3 c shows an example of how the energy of a sampleimage is distributed before and after a phase-modulation stage in whicha pseudo-random phase distribution is used. It can be seen thatmodulating an image by such a phase distribution has the effect ofredistributing the energy more evenly throughout the spatial-frequencydomain. The skilled person will appreciate that there are many ways inwhich pseudo-random binary-phase modulation data may be generated (forexample, a shift register with feedback).

The quantisation block takes complex hologram data, which is produced asthe output of the preceding space-frequency transform block, and maps itto a restricted set of values, which correspond to actual modulationlevels that can be achieved on a target SLM (the different quantisedphase retardation levels may need not have a regular distribution). Thenumber of quantisation levels may be set at two, for example for an SLMproducing phase retardations of 0 or π at each pixel.

In embodiments the quantiser is configured to separately quantise realand imaginary components of the holographic sub-frame data to generate apair of holographic sub-frames, each with two (or more)phase-retardation levels, for the output buffer. FIG. 3 d shows anexample of such a system. It can be shown that for discretely pixellatedfields, the real and imaginary components of the complex holographicsub-frame data are uncorrelated, which is why it is valid to treat thereal and imaginary components independently and produce two uncorrelatedholographic sub-frames.

An example of a suitable binary phase SLM is the SXGA (1280×1024)reflective binary phase modulating ferroelectric liquid crystal SLM madeby CRL Opto (Forth Dimension Displays Limited, of Scotland, UK). Aferroelectric liquid crystal SLM is advantageous because of its fastswitching time. Binary phase devices are convenient but some preferredembodiments of the method use so-called multiphase spatial lightmodulators as distinct from binary phase spatial light modulators (thatis SLMs which have more than two different selectable phase delay valuesfor a pixel as opposed to binary devices in which a pixel has only oneof two phase delay values). Multiphase SLMs (devices with three or morequantized phases) include continuous phase SLMs, although when driven bydigital circuitry these devices are necessarily quantised to a number ofdiscrete phase delay values. Binary quantization results in a conjugateimage whereas the use of more than binary phase suppresses the conjugateimage (see WO 2005/059660).

Adaptive OSPR

In the OSPR approach we have described above subframe holograms aregenerated independently and thus exhibit independent noise. In controlterms, this is an open-loop system. However one might expect that betterresults could be obtained if, instead, the generation process for eachsubframe took into account the noise generated by the previous subframesin order to cancel it out, effectively “feeding back” the perceivedimage formed after, say, n OSPR frames to stage n+1 of the algorithm. Incontrol terms, this is a closed-loop system.

One example of this approach comprises an adaptive OSPR algorithm whichuses feedback as follows: each stage n of the algorithm calculates thenoise resulting from the previously-generated holograms H₁ to H_(n−1)and factors this noise into the generation of the hologram H_(n) tocancel it out. As a result, it can be shown that noise variance falls as1/N². An example procedure takes as input a target image T, and aparameter N specifying the desired number of hologram subframes toproduce, and outputs a set of N holograms H₁ to H_(N) which, whendisplayed sequentially at an appropriate rate, form as a far-field imagea visual representation of T which is perceived as high quality:

An optional pre-processing step performs gamma correction to match a CRTdisplay by calculating T(x, y)^(1.3). Then at each stage n (of N stages)an array F (zero at the procedure start) keeps track of a “runningtotal” (desired image, plus noise) of the image energy formed by theprevious holograms H₁ to H_(n−1) so that the noise may be evaluated andtaken into account in the subsequent stage: F(x, y):=F(x,y)+|F[H_(n−1)(x, y)]|². A random phase factor φ is added at each stageto each pixel of the target image, and the target image is adjusted totake the noise from the previous stages into account, calculating ascaling factor α to match the intensity of the noisy “running total”energy F with the target image energy (T′)². The total noise energy fromthe previous n−1 stages is given by αF−(n−1)(T′)², according to therelation

$\alpha:=\frac{\sum\limits_{x,y}{T^{\prime}( {x,y} )}^{4}}{\sum\limits_{x,y}{{F( {x,y} )} \cdot {T^{\prime}( {x,y} )}^{2}}}$and therefore the target energy at this stage is given by the differencebetween the desired target energy at this iteration and the previousnoise present in order to cancel that noise out, i.e.(T′)²−[αF−(n−1)(T′)²]=n(T′)²+αF. This gives a target amplitude |T″|equal to the square root of this energy value, i.e.

${T^{''}( {x,y} )}:=\{ \begin{matrix}{{\sqrt{{2{T^{\prime}( {x,y} )}^{2}} - {\alpha\; F}} \cdot \exp}\{ {j\;{\phi( {x,y} )}} \}} & {{{if}\mspace{14mu} 2{T^{\prime}( {x,y} )}^{2}} > {\alpha\; F}} \\0 & {otherwise}\end{matrix} $

At each stage n, H represents an intermediate fully-complex hologramformed from the target T″ and is calculated using an inverse Fouriertransform operation. It is quantized to binary phase to form the outputhologram H_(n), i.e.

H(x, y) := F⁻¹[T^(″)(x, y)]${H_{n}( {x,y} )} = \{ \begin{matrix}1 & {{{if}\mspace{14mu}{{Re}\lbrack {H( {x,y} )} \rbrack}} > 0} \\{- 1} & {otherwise}\end{matrix} $FIG. 4 a outlines this method and FIG. 4 b shows details of an exampleimplementation, as described above.

Thus, broadly speaking, an ADOSPR-type method of generating data fordisplaying an image (defined by displayed image data, using a pluralityof holographically generated temporal subframes displayed sequentiallyin time such that they are perceived as a single noise-reduced image),comprises generating from the displayed image data holographic data foreach subframe such that replay of these gives the appearance of theimage, and, when generating holographic data for a subframe,compensating for noise in the displayed image arising from one or moreprevious subframes of the sequence of holographically generatedsubframes. In embodiments the compensating comprises determining a noisecompensation frame for a subframe; and determining an adjusted versionof the displayed image data using the noise compensation frame, prior togeneration of holographic data for a subframe. In embodiments theadjusting comprises transforming the previous subframe data from afrequency domain to a spatial domain, and subtracting the transformeddata from data derived from the displayed image data.

More details, including a hardware implementation, can be found inWO2007/141567 hereby incorporated by reference.

Colour Holographic Image Projection

The total field size of an image scales with the wavelength of lightemployed to illuminate the SLM, red light being diffracted more by thepixels of the SLM than blue light and thus giving rise to a larger totalfield size. Naively a colour holographic projection system could beconstructed by superimposed simply three optical channels, red, blue andgreen but this is difficult because the different colour images must bealigned. A better approach is to create a combined beam comprising red,green and blue light and provide this to a common SLM, scaling the sizesof the images to match one another.

FIG. 5 a shows an example colour holographic image projection system1000, here including demagnification optics 1014 which project theholographically generated image onto a screen 1016. Since the image isgenerated holographically it is in focus at substantially all distancesfrom the optics 1014. The system comprises red 1002, green 1006, andblue 1004 collimated laser diode light sources, for example atwavelengths of 638 nm, 532 nm and 445 nm, driven in a time-multiplexedmanner. Each light source comprises a laser diode 1002 and, ifnecessary, a collimating lens and/or beam expander. Optionally therespective sizes of the beams are scaled to the respective sizes of theholograms, as described later. The red, green and blue light beams arecombined in two dichroic beam splitters 1010 a, b and the combined beamis provided (in this example) to a reflective spatial light modulator1012; the figure shows that the extent of the red field would be greaterthan that of the blue field. The total field size of the displayed imagedepends upon the pixel size of the SLM but not on the number of pixelsin the hologram displayed on the SLM.

FIG. 5 b shows padding an initial input image with zeros in order togenerate three colour planes of different spatial extents for blue,green and red image planes. A holographic transform is then performed onthese padded image planes to generate holograms for each sub-plane; theinformation in the hologram is distributed over the complete set ofpixels. The hologram planes are illuminated, optionally bycorrespondingly sized beams, to project different sized respectivefields on to the display screen. FIG. 5 c shows upsizing the inputimage, the blue image plane in proportion to the ratio of red to bluewavelength (638/445), and the green image plane in proportion to theratio of red to green wavelengths (638/532) (the red image plane isunchanged). Optionally the upsized image may then be padded with zerosto a number of pixels in the SLM (preferably leaving a little spacearound the edge to reduce edge effects). The red, green and blue fieldshave different sizes but are each composed of substantially the samenumber of pixels, but because the blue, and green images were upsizedprior to generating the hologram a given number of pixels in the inputimage occupies the same spatial extent for red, green and blue colourplanes. Here there is the possibility of selecting an image size for theholographic transform procedure which is convenient, for example amultiple of 8 or 16 pixels in each direction.

Super-Resolution ADOSPR

In a 2D holographic video projection system, the output resolution isnormally at most the resolution of the microdisplay, because the outputimage in the replay field is the Fourier transform of the hologram onthe microdisplay (a bijective mapping from X^(M×M) to X^(M×M)). Further,when a binary-phase modulator is employed as the microdisplay, with sayM×M-pixels, the presence of the conjugate image restricts theaddressable output resolution to M×M/2 points. However the inventorshave recognised that inter-pixel interference may be exploited toproduce increased resolution: Each point in the output is a copy of theFourier transform of the hologram aperture (if, say, the aperture issquare and the illumination uniform this corresponds to a sinc-shapedpixel in the output). The main lobe of such a sinc function is widerthan the inter-pixel distance in the output and therefore adjacentpixels will interfere with one another. Ordinarily this is detrimentalbut it is possible to exploit, the effect to advantage.

The eye perceives not the field amplitude F but its intensity |F|² andthus manipulation of the phases allows one to influence the pixel valuesbetween the sampling grid to create structure at increased spatialfrequencies. Super-resolution can be implemented using an ADOSPR-typeprocedure to generate OSPR hologram sets of resolution M×M that formimage reproductions at double (in each dimension) the resolution of thatof the hologram, i.e. 2M×2M (2M×M for a binary phase modulator).

We extend the ADOSPR procedure so that, in addition to feeding forwardthe reproduction error present at each of the M×M sampling points (x,y), the errors present between the sampling points after stage N−1, i.e.at (x½, y), (x, y½) and (x½, y½), are also fed forwards and compensatedfor when calculating the hologram H_(N) in stage N. In embodiments thisuses a modified inter-pixel Fourier transform operation to evaluate thefrequency components every half-sample, instead of every sample. As analternative to half-sample evaluation, such a transform can beimplemented by, for example, padding each M×M hologram up to 2M×2M byembedding it in a matrix of zeros; in either case and we notate this asF^(2M×2M)[H(x, y)]. Taking the Fourier transform of this padded hologramthen produces a 2M×2M field, which can be adjusted for error as desiredbefore taking the inverse Fourier transform to obtain a 2M×2M hologram,which is then bandlimited to form the next M×M hologram in the outputOSPR set.

Because the hologram is the frequency spectrum of the image, phaseholograms (which have uniform amplitude everywhere) form images with auniform, flat frequency spectrum. For a fixed amplitude target imagethis implies a requirement of effectively random phase in the imagepixels, which would appear to be incompatible with using inter-pixelinterference. However an OSPR-with-feedback approach allows phasecontrol to be achieved over a set of subframe holograms eachindividually having a substantially flat, pseudorandom phase spectrum.In an example super-resolution OSPR-with-feedback procedure thevariables are as follows:

-   -   N is the number of OSPR subframes to generate.    -   T is the input video frame of resolution 2M×2M.    -   The M×M-pixel holograms H₁ . . . H_(N) produced at the end of        each stage form the output OSPR hologram set.    -   At each stage of the algorithm, φ(x, y) is re-initialised to a        2M×2M array of uniformly-distributed random phases. Q iterations        of a coherent optimisation sub-algorithm are employed to adjust        these phases towards an error minimum.    -   F(x, y) holds a dynamically-updated 2M×2M-pixel reconstruction        of the effect of the hologram subframes calculated so far.    -   γ is the desired display output gamma (2.2 corresponds roughly        to a standard CRT).

We next make the following definitions:

Input X Output Y Operator Description size size Definition F Fouriertransform 2M × 2M 2M × 2M${Y( {u,v} )} = {\sum\limits_{x = {{- M} + 1}}^{M}\;{\sum\limits_{y = {{- M} + 1}}^{M}\; e^{{- 2}\;\pi\;{j{(\frac{{ux} + {vy}}{2M})}}}}}$F⁻¹ Inverse Fourier transform 2M × 2M 2M × 2M${Y( {u,v} )} = {\sum\limits_{x = {{- M} + 1}}^{M}\;{\sum\limits_{y = {{- M} + 1}}^{M}\; e^{2\;\pi\;{j{(\frac{{ux} + {vy}}{2M})}}}}}$F^(2M×2M) Inter-pixel Fourier transform M × M 2M × 2M${Y( {u,v} )} = {\sum\limits_{x = {{- \frac{M}{2}} + 1}}^{\frac{M}{2}}\;{\sum\limits_{y = {{- \frac{M}{2}} + 1}}^{\frac{M}{2}}\; e^{{- 2}\;\pi\;{j{(\frac{{ux} + {vy}}{2M})}}}}}$The modified (inter-pixel) Fourier transform effectively evaluates aFourier (or inverse Fourier) transform at intermediate image points i.e.

$ f_{0,0}arrow{\begin{matrix}F_{0,0} \\F_{0,0.5}\end{matrix}\begin{matrix}F_{0.5,0} \\F_{0.5,0.5}\end{matrix}} , f_{1,0}arrow{\begin{matrix}F_{1,0} \\F_{1,0.5}\end{matrix}\begin{matrix}F_{1.5,0} \\F_{1.5,0.5}\end{matrix}} ,{\ldots\;.}$

FIG. 6 a shows an outline of the procedural steps which are described indetail below.

Preprocessing

${T^{\prime}( {x,y} )}:={{T( {x,y} )}^{\frac{\gamma}{2}}\mspace{14mu}({optional})}$

Stage 1

  F(x, y) := 0   T^(″)(x, y) := T^(′)(x, y) ⋅ exp {j ϕ(x, y)}$\mspace{34mu}{{iterate}\mspace{14mu} Q\mspace{14mu}{{times}\mspace{14mu}\lbrack \begin{matrix}{{H^{''}( {x,y} )}:={F^{- 1}\lbrack {T^{''}( {x,y} )} \rbrack}} \\{{H^{\prime}( {x,y} )}:=\{ \begin{matrix}1 & {{{if}\mspace{14mu}{{Re}\lbrack {H^{''}( {x,y} )} \rbrack}} > 0} \\{- 1} & {otherwise}\end{matrix} } \\{{H_{1}( {x,y} )}:={H^{\prime}( {{{- \frac{M}{2}} \leq x < \frac{M}{2}},{- \frac{M}{2}},{\leq y < \frac{M}{2}}} )}} \\{{X( {x,y} )} = {F^{2\; M \times 2\; M}\lbrack {H_{1}( {x,y} )} \rbrack}} \\{{T^{''}( {x,y} )} = {{{T^{\prime}( {x,y} )} \cdot \exp}\{ {{j\angle}\;{X( {x,y} )}} \}}}\end{matrix} }}$

Stage 2

  F(x, y) := F(x, y) + F^(2 M × 2 M)[H₁(x, y)]²$\mspace{20mu}{\alpha:=\frac{\sum\limits_{x,y}{T^{\prime}( {x,y} )}^{4}}{\sum\limits_{x,y}{{F( {x,y} )} \cdot {T^{\prime}( {x,y} )}^{2}}}}$$\mspace{25mu}{{T^{''}( {x,y} )}:=\{ {\begin{matrix}{{\sqrt{{2\;{T^{\prime}( {x,y} )}^{2}} - {\alpha\; F}} \cdot \exp}\{ {j\;{\phi( {x,y} )}} \}} & {{{if}\mspace{14mu} 2{T^{\prime}( {x,y} )}^{2}} > {\alpha\; F}} \\0 & {otherwise}\end{matrix}\mspace{25mu}{iterate}\mspace{14mu} Q\mspace{14mu}{{times}\mspace{14mu}\lbrack \begin{matrix}{{H^{''}( {x,y} )}:={F^{- 1}\lbrack {T^{''}( {x,y} )} \rbrack}} \\{{H^{\prime}( {x,y} )}:=\{ \begin{matrix}1 & {{{if}\mspace{14mu}{{Re}\lbrack {H^{''}( {x,y} )} \rbrack}} > 0} \\{- 1} & {otherwise}\end{matrix} } \\{{H_{2}( {x,y} )}:={H^{\prime}( {{{- \frac{M}{2}} \leq x < \frac{M}{2}},{- \frac{M}{2}},{\leq y < \frac{M}{2}}} )}} \\{{X( {x,y} )} = {F^{2\; M \times 2\; M}\lbrack {H_{2}( {x,y} )} \rbrack}} \\{{T^{''}( {x,y} )} = {{{T^{\prime}( {x,y} )} \cdot \exp}\{ {{j\angle}\;{X( {x,y} )}} \}}}\end{matrix} }} }$

Note that in the above F(x,y) is different to the transform or inversetransform F (which has a superscript).

Stage N

$\begin{matrix}{\mspace{79mu}{{F( {x,y} )}:={{F( {x,y} )} + {{F^{2\; M \times 2\; M}\lbrack {H_{N - 1}( {x,y} )} \rbrack}}^{2}}}} & {{update}\mspace{14mu}{dynamic}\mspace{14mu}{output}\mspace{14mu}{estimate}} \\ \begin{matrix}{\alpha:=\frac{( {N - 1} ){\sum\limits_{x,y}{T^{\prime}( {x,y} )}^{4}}}{\sum\limits_{x,y}{{F( {x,y} )} \cdot {T^{\prime}( {x,y} )}^{2}}}} \\{{T^{''}( {x,y} )}:=\{ \begin{matrix}{{\sqrt{{N \cdot {T^{\prime}( {x,y} )}^{2}} - {\alpha\; F}} \cdot \exp}\{ {j\;{\phi( {x,y} )}} \}} & {{{if}\mspace{14mu}{N \cdot {T^{\prime}( {x,y} )}^{2}}} > {\alpha\; F}} \\0 & {otherwise}\end{matrix} }\end{matrix} \} & \begin{matrix}{{calculate}\mspace{14mu} 2M \times 2\; M\mspace{14mu}{noise}} \\{{compensation}\mspace{14mu}{target}}\end{matrix} \\{{iterate}\mspace{14mu} Q\mspace{14mu}{{times}\mspace{14mu}\lbrack \begin{matrix}{{H^{''}( {x,y} )}:={F^{- 1}\lbrack {T^{''}( {x,y} )} \rbrack}} \\{{H^{\prime}( {x,y} )}:=\{ \begin{matrix}1 & {{{if}\mspace{14mu}{{Re}\lbrack {H^{''}( {x,y} )} \rbrack}} > 0} \\{- 1} & {otherwise}\end{matrix} } \\{{H_{N}( {x,y} )}:={H^{\prime}( {{{- \frac{M}{2}} \leq x < \frac{M}{2}},{- \frac{M}{2}},{\leq y < \frac{M}{2}}} )}} \\{{X( {x,y} )} = {F^{2\; M \times 2\; M}\lbrack {H_{N}( {x,y} )} \rbrack}} \\{{T^{''}( {x,y} )} = {{{T^{\prime}( {x,y} )} \cdot \exp}\{ {{j\angle}\;{X( {x,y} )}} \}}}\end{matrix} }} & \begin{matrix}{{{calculate}\mspace{14mu} M \times M} - {bandlimited}} \\{{binary}\mspace{14mu}{hologram}\mspace{14mu} H_{N}} \\( {{other}\mspace{14mu}{approaches}\mspace{14mu}{may}\mspace{14mu}{be}\mspace{14mu}{used}} )\end{matrix}\end{matrix}$

Referring to FIG. 6 b, this shows a detailed block diagram of a systemfor generating a plurality (N) of subframe holograms for displaying aresolution-enhanced image according to the above procedure. In theFigure the operations described above are associated with arrows and theresulting data (typically a two dimensional matrix) by blocks in which

denotes,

complex valued data, and {−1,1} quantized (here binarised) data. Thevariables associated with the 2D matrices are shown alongside theblocks, and the dimensions of the matrices are indicated by arrows. Inthe Figure the blocks (matrices) are square but rectangular imagematrices may also be used.

For further details reference may be made to WO 2007/085874, herebyincorporated by reference.

Sub-Segment OSPR

Broadly this involves subdividing the replay field into a plurality ofspatially interlaced regions, and displaying holograms for each of theinterlaced regions such that interference between adjacent pixels of thereplay field is reduced. Thus each interlaced region may comprise a setof pixels of the replay field in which each pixel is surrounded bypixels of substantially zero light intensity, for example regular gridswith spaces in between. Holograms for the interlaced regions may bedisplayed with calculated phase shifts to provide lateral displacements(in pixels) in said replay field of: (0,0), (0,1), (1,0), and (1,1).

A M×M-pixel image T (or a single colour plane for a full-colour system)is divided into multiple interlaced regions, for example by selecting(x,y) coordinate pixels with even-even, even-odd, odd-even, and odd-oddcoordinates. The subfield images are termed T⁰⁰, T⁰¹, T¹⁰, and T¹¹, asfollows:T _(yx) ⁰⁰ =T _(2y,2x) T _(yx) ⁰¹ =T _(2y,2x+1)T _(yx) ¹⁰ =T _(2y+1,2x) T _(yx) ¹¹ =T _(2y+1,2x+1)

-   -   T⁰⁰ contains the image pixels with even x and even y coordinates    -   T⁰¹ contains the image pixels with odd x and even y coordinates    -   T¹⁰ contains the image pixels with even x and odd y coordinates    -   T¹¹ contains the image pixels with odd x and odd y coordinates

For each of these sub-segments, the corresponding

${\frac{M}{2} \times \frac{M}{2}} - {pixel}$subfield holograms H₀₀, H₀₁, H₁₀ and H₁₁ are then calculated. A variantof our ADOSPR algorithm we term SSOSPR (sub-segment OSPR) may be used tocalculate the N sub-frames of the subfield holograms (as described indetail below).

The full (pseudo-replicated) hologram H′₀₀, H′₀₁, H′₁₀ and H′₁₁ of eachinterlaced region is formed by displaying a plurality of substantialreplicas of said subfield hologram simultaneously on said SLM. H′₀₀,H′₀₁, H′₁₀ and H′₁₁ are defined as follows:

$H_{00}^{\prime} = \begin{pmatrix}H_{00} & H_{00} \\H_{00} & H_{00}\end{pmatrix}$ $H_{01}^{\prime} = \begin{pmatrix}H_{10} & {- H_{10}} \\H_{10} & {- H_{10}}\end{pmatrix}$ $H_{10}^{\prime} = \begin{pmatrix}H_{01} & H_{01} \\{- H_{01}} & {- H_{01}}\end{pmatrix}$ $H_{11}^{\prime} = \begin{pmatrix}H_{11} & {- H_{11}} \\{- H_{11}} & H_{11}\end{pmatrix}$H′₀₀ comprises four replicas of the subfield hologram H₀₀ one in eachquadrant (or tile) of the hologram. Displaying H′₀₀ on the display willrender just the image pixels with even x and even y coordinates in thecorrect locations, with zeroes elsewhere in the reproduction. Similarly,H′₀₁ comprises four replicas of the subfield hologram H₀₁ one in eachquadrant of the hologram with the data inverted for the right-handquadrants. H′₀₁ will render just the image pixels with odd x and even ycoordinates, with zeroes elsewhere, and so forth. As a result, if wetime-sequence the holograms H′₀₀, H′₀₁, H′₁₀, H′₁₁, the entire image isformed through incoherent summation (in the eye) of the four interlacedregions.

Before processing each subfield image, four complex phase shiftmatrices, along with their complex conjugates, are computed. Theseprovide lateral displacements of (0,0), (0,1), (1,0), and (1,1) pixelsin the replay field. The matrices are fixed and can in principle bepre-stored, or their elements generated on-the-fly as data passesthrough the FFT engine.

These phase shift matrices are of size

$\frac{M}{2} \times \frac{M}{2}$and have elements given below.

$\begin{matrix}{P_{vu}^{00} = 1} & {P_{vu}^{01} = {\mathbb{e}}^{\frac{2\;\pi\; j\; u}{M}}} \\{P_{vu}^{10} = {\mathbb{e}}^{\frac{2\;\pi\; j\; v}{M}}} & {P_{vu}^{11} = {\mathbb{e}}^{\frac{2\;{{\pi j}{({u + v})}}}{M}}}\end{matrix}\mspace{50mu}\begin{matrix}{{\overset{\_}{P}}_{vu}^{00} = 1} & {{\overset{\_}{P}}_{vu}^{01} = {\mathbb{e}}^{\frac{{- 2}\;{\pi j}\; u}{M}}} \\{{\overset{\_}{P}}_{vu}^{10} = {\mathbb{e}}^{\frac{{- 2}\;\pi\; j\; v}{M}}} & {{\overset{\_}{P}}_{vu}^{11} = {\mathbb{e}}^{\frac{{- 2}\;{{\pi j}{({u + v})}}}{M}}}\end{matrix}$where

${0 \leq v},{u < \frac{M}{2}},$with v representing the vertical coordinate in hologram space, and urepresenting the horizontal coordinate.

Each of the target segments T^(qp) may be processed independently, inparallel or (probably preferably) sequentially. We define terms asfollows:

-   -   The loop variable i represents the current sub-frame number    -   T_(yx) ^(qp) represents the amplitude of the input image for        sub-segment qp, at coordinates (x, y)    -   T_(yx) ^((l,qp)) represents the target image energy of sub-frame        i, sub-segment qp    -   E_(yx) ^(qp) represents the constantly-updated estimate of the        reconstruction field intensity error    -   {circumflex over (T)}_(yx) ^((i,qp)) represents the desired        target image field, adjusted for the intensity error E present    -   H_(vu) ^((i,qp)) and Ĥ_(vu) ^((i,qp)) represent non-quantised        and quantised holograms respectively, generated by the algorithm    -   P^(qp) represent the phase-shift matrices described above, with        P ^(qp) representing their complex conjugates    -   The loop variable q represents the iteration number of the        coherent optimisation loop (Liu-Taghizadeh)    -   ψ_(yx) ^((i,qp)) represents the fully-complex reconstruction        field    -   ψ′_(yx) ^((i,qp)) represents an error-reducing modification to        the reconstruction field ψ_(xy) ^((i,qp))    -   I_(yx) ^((i,qp)) represents the instantaneous intensity of the        reconstruction field, as perceived by the eye    -   α and κ represent Fourier transform scaling constants    -   γ₁ and γ₂ represent fixed algorithm constants, with final values        to be determined (currently γ₁=2 and γ₂=1)

The algorithm that follows is executed in its entirety for each segmentqp, where qp is 00, 01, 10 or 11: The first step of initialisingalgorithm variables sets:

i := 1${T_{yx}^{({1,{qp}})}:={{\lbrack T_{yx}^{qp} \rbrack^{2}\mspace{11mu} 0} \leq y}},{x < \frac{M}{2}}$E_(yx)^(qp) := 0The field error estimate E is initially set to zero and refined in lateriterations.

The target field, adjusted for field error E, is calculated as:

${\hat{T}}_{yx}^{({i,{qp}})}:=\{ \begin{matrix}\sqrt{T_{yx}^{({i,{qp}})} - E_{yx}^{qp}} & {{{if}\mspace{14mu} T_{yx}^{({i,{qp}})}} > E_{yx}^{qp}} \\0 & {otherwise}\end{matrix} $A first approximation to the hologram is generated by phase modulatingthe target field, i.e. multiplying by e^(jθ), then transforming, i.e. byapplying an inverse Fourier transform and multiplying pointwise with theappropriate phase-shift matrix P. The steps of phase modulating andtransforming are as used in the OSPR procedure described above. Thevariation is the introduction of multiplication by phase-shift matrix P.H _(vu) ^((i,qp)) :=P _(vu) ^(qp) .F ⁻¹ [{circumflex over (T)} _(yx)^((i,qp)) .e ^(jφ) ^((i)) _(yx ])

As in the OSPR procedure described above, the fully complex hologram maythen optionally quantised to binary phase, as follows:

${\hat{H}}_{vu}^{({i,{qp}})}:=\{ \begin{matrix}{- 1} & {{{Re}( H_{vu}^{({i,{qp}})} )} \leq 0} \\1 & {{{Re}( H_{vu}^{({i,{qp}})} )} > 0}\end{matrix} $Although binary phase quantisation is described, multi-phasequantisation is an alternative approach. The first approximation may berefined, e.g. by applying the four steps of the Liu-Taghizadeh algorithmor another similar sub-algorithm.

Step 1 of the Liu-Taghizadeh algorithm is to calculate the FFT of theith hologram, e.g. by applying the following equations for the binarisedhologram:q:=0ψ_(yx) ^((i,qp)) :=F[ P _(vu) ^(qp) .Ĥ _(vu) ^((i,qp))]

Step 2 is to update the obtained field with coherent noise compensationin the specified signal window W. Constants are γ₁=2, γ₂=1 (subject tochange)

$\kappa:=\frac{\sum\limits_{x,{y \in W}}{\psi_{yx}^{({i,{qp}})}}}{\sum\limits_{x,{y \in W}}\sqrt{{\hat{T}}_{yx}^{({i,{qp}})}}}$$\psi_{yx}^{\prime{({i,{qp}})}}:=\{ \begin{matrix}{{{{\gamma_{1}\kappa\sqrt{{\hat{T}}_{yx}^{({i,{qp}})}}} - {\gamma_{2}\psi_{yx}^{({i,{qp}})}}}} \cdot {\mathbb{e}}^{{j\angle\psi}_{yx}^{({i,{qp}})}}} & {( {x,y} ) \in W} \\{unchanged} & {( {x,y} ) \notin W}\end{matrix} $

Step 3 is to calculate a first iteration of an improved hologram whichmay then be optionally binarised:

H_(vu)^((i, qp)) := P_(vu)^(qp) ⋅ F⁻¹[ψ_(yx)^(′(i, qp))]${\hat{H}}_{vu}^{({i,{qp}})}:=\{ \begin{matrix}{- 1} & {{{Re}( H_{vu}^{({i,{qp}})} )} \leq 0} \\1 & {{{Re}( H_{vu}^{({i,{qp}})} )} > 0}\end{matrix} $

Step 4 is to complete the next loop of the Liu-Taghizadeh sub-algorithm,feeding each iteration of the improved hologram through, until done Qiterations have been completed, i.e.q=q+1

Go to step 1 of the Liu-Taghizadeh algorithm if q<Q

The hologram generated by the final iteration of the Liu-Taghizadehsub-algorithm is then sent to the display. The Liu-Taghizadehsub-algorithm is a standard sub-algorithm that may be replaced withequivalent sub-algorithms; it is not essential that such a sub-algorithmis used. The Liu-Taghizadeh sub-algorithm may be altered to generate animproved hologram when the binarisation step is omitted.

Once the Liu-Taghizadeh sub-algorithm is completed, the intensity of thehologram is multiplied by the complex conjugate of the phase-shiftmatrix and its Fast Fourier transform is calculated. In other words, thetransform and phase-shift steps detailed above are reversed.I _(yx) ^((i,qp)) :=|F[ P _(vu) ^(qp) .Ĥ _(vu) ^((i,qp))]|²The equation above shows a binarised hologram but the equation may beamended to calculate the intensity for a hologram which has not beenbinarised.

The intensity error estimate is calculated to compensate for the noiseperceived by the eye at this point as follows:

$\alpha:=\sqrt{\frac{\sum\limits_{x,y}{T_{yx}^{({i,{qp}})}I_{yx}^{({i,{qp}})}}}{\sum\limits_{x,y}\lbrack I_{yx}^{({i,{qp}})} \rbrack^{2}}}$E_(yx)^(qp) := E_(yx)^(qp) + α I_(yx)^((i, qp)) − T_(yx)^((i, qp))The calculated intensity error is fed into the second step, namelycalculate the target field and all the subsequent steps of the algorithmare re-calculated. The algorithm loops until all N holograms have beenproduced.

The algorithm is run for each of the 4 sub-segments, generating a totalof 4N hologram sub-frames, given by H_(vu) ^((i,qp)). These hologramsare then processed (preferably internally in the display) to form thepseudo-replicated holograms H′_(vu) ^((i,qp)) defined above, which arethen displayed.

Because the subfield holograms are independent, they can be computedsequentially. As each subfield hologram requires a Fourier transform ofsize of only

${\frac{M}{2} \times \frac{M}{2}},$instead of M×M, the memory size required for the transform step isreduced by a factor of four. The above procedure is relativelycomputationally complex, but because the techniques we describe lateremploy only a relatively low resolution hologram SLM this need not be aproblem. However for video the concepts of i-frames and s-frames may beemployed (an initial frame represents a new incoming video frame forwhich holograms are generated from scratch; a subsequent frame uses thehologram generated for the previous video frame as an initial estimate).

For further details reference may be made to GB 0724161.5 filed 11 Dec.2007, hereby incorporated by reference.

Dual Modulation Architecture

We now describe an improved architecture which employs dual SLMmodulation—low resolution phase modulation and higher resolutionamplitude (intensity) modulation. This can provide substantialimprovements over the approaches we have previously described, inparticular improvements in one or more of: image quality, resolution,contrast, brightness, power consumption and physical size.

Most of the energy of a typical video image is concentrated in the lowspatial frequencies. This is illustrated in FIG. 1 which shows anexample image (left) and the corresponding power spectrum (right).

Since the primary gain of holographic projection over imaging is one ofenergy efficiency, one can reason that it is only the low spatialfrequencies of an image that need to be rendered holographically tomaintain high efficiency. Because the resolution of a hologramdetermines the maximum spatial frequency that can be represented in thecorresponding image, it follows that only a very low-resolution hologramis required to accurately render the low spatial frequencies of a videoimage, which represent most of its energy. The high-frequency componentscan then be rendered with an intensity-modulating imaging panel, placedin a plane conjugate to the hologram SLM. Effectively, diffracted lightfrom the hologram SLM device (SLM1) is used to illuminate the imagingSLM device (SLM2). Because the high-frequency components containrelatively little energy, the light blocked by the imaging SLM does notsignificantly decrease the efficiency of the system, unlike in aconventional imaging system.

The hologram SLM should preferably be a fast multi-phase device, forhigh diffraction efficiency. Successful results have been obtained insimulation with the imaging SLM being either a fast binary device (FLC)or a slow analogue device (nematic). Such an approach has a number ofattractive features as we have previously mentioned. These includesignificantly lower power consumption due to reduced computation; asystem which is scalable with resolution: no new ASIC or hologram SLMrequired when it is desired to increase projector resolution; theability to use existing high-resolution FLC (Ferroelectric LiquidCrystal)/nematic panels or even a DLP (digital light processor for theimaging SLM; an increased ANSI contrast (potentially greater than1000:1, 1500:1 or 2000:1—an order of magnitude high than achievable withan imaging SLM alone); and a significant size reduction as the laserbeams now illuminate a significantly smaller hologram SLM, substantiallyshrinking down the illumination optics. Embodiments of the system alsohave relatively slow PWM (Pulse Width Modulation) laser modulation (e.g.1-2 kHz). Apart, in embodiments for a very small motion of a diffuser,no moving parts used to form an image, so they system can be robust toshock and vibration; 8 bit intensity resolution is possible.

FIG. 2 shows a reference optical layout for a holographic imageprojection system 200 an embodiment of the invention. In the full colourholographic image projector of FIG. 2 there are red R, green G, and blueB lasers. The system also includes the following additional elements:

-   -   SLM1 is the hologram SLM (spatial light modulator), potential        size 32×32 or 64×64 pixels, of pixel pitch A.    -   L1, L2 and L3 are collimation lenses for the R, G and B lasers        respectively. For a 64×64 pixel hologram SLM with 5.62 μm        pixels, the SLM active area is around 0.36 mm×0.36mm, so it        should be possible to employ a very slow illuminator design.    -   M1, M2 and M3 are the corresponding dichroic mirrors. Again,        these need be only slightly larger than the laser beam waist        (0.36 mm).    -   PBS1 (Polarising Beam Splitter 1) transmits the incident        illumination to SLM1. Diffracted light produced by        SLM1—naturally rotated (with a liquid crystal SLM) in        polarisation by 90 degrees—is then reflected by PBS1 towards L4.        PBS1 needs to have a clear aperture at least as large as the        active area of SLM1.    -   SLM2 is the imaging SLM, of size equal to the target image        resolution (e.g. 854×480).    -   Lens L4 forms an intermediate image plane on the surface of        SLM2. Its focal length f is set so that fλ/Δ is equal to the        size of the active area of the imaging SLM. That is the        intermediate real image from the hologram(s) on SLM1 fits        on/covers the active area of SLM2 which modulates this image. In        embodiments L4 may be encoded into the hologram(s) on SLM1, for        example using the techniques we have described in WO2007/110668.    -   PBS2 (Polarising Beam Splitter 2) transmits incident light to        SLM2, and reflects emergent light into the path of the output        optics. PBS2 should have a clear aperture at least as large as        the active area of SLM2.    -   Lenses L5 and L6 form an output telescope (demagnifying optics),        as with holographic projectors we have previously described. The        output projection angle is proportional to the ratio of the        focal length of L5 to that of L6.    -   D1 is a piezoelectrically-actuated diffuser to reduce speckle,        as we have described, for example in GB0800167.9.

A system controller 202 performs signal processing in either dedicatedhardware, or in software, or in a combination of the two, as describedfurther below. Thus controller 202 inputs image data and provides lowspatial frequency hologram data 204 to SLM1 and higher spatial frequencyintensity modulation data 206 to SLM2. The controller also provideslaser light intensity control data 208 to each of the three lasers.

In embodiments SLM1 may be a reflective charge-driven 90° ferroelectricliquid crystal SLM from DisplayTech® with a quarter wave plate betweenthe pixel mirror layer and the liquid crystal material. Alternatively aMirasol® SLM from Qualcomm Inc may be used.

EXAMPLE PROCEDURE

We now describe an example procedure to implement embodiments of theinvention. This example procedure is based on super-resolution ADOSPRbut approaches based, for example, on ADOSPR and on sub-segment ADOSPRmay also be employed. In general the techniques are not limited to usewith an OSPR-type hologram generation procedure, although, this iscomputationally efficient. The procedure assumes a fast phase-modulating(binary or multi-phase) hologram SLM, and a (slower) nematic imagingSLM, although the skilled person will appreciate that other imagingtechnologies may be equally appropriate.

In all cases the illumination incident on the SLM is assumed to beGaussian, with the 1/e² intensity at the edges of the SLM.

Variables

-   -   1. The hologram SLM size is M×M pixels.    -   2. The input image target amplitude, T, is of size P×P pixels.        Amplitude range for the input is between 0 (black) and 1        (white).    -   3. N ADOSPR subframes are to be generated.    -   4. D is a diffraction efficiency boost parameter controlling the        trade-off between reconstruction error and diffraction        efficiency A value of 1.0 gives theoretically perfect        reconstruction; larger values of D increase the optical        efficiency at the expense of increasing the noise. Based on the        appearance of simulated images a practical maximum for D appears        to be less than 2, for example approximately 1.5 (see below).

Procedure Flow

-   -   1. Form a 2M×2M target image, R, for hologram generation        comprising peak values of blocks of the image. Subdivide the        input (P×P) image T into 2M×2M blocks, each of size P/2M×P/2M.        Set each pixel of the target R to be the peak amplitude of the        image data within the corresponding P/2M×P/2M block of the        image.    -   2. Generate a hologram set H of N holograms of size M×M from R.        In this example, the above-described super-resolution ADOSPR        algorithm is employed, optionally iteratively optimising the        holograms, for example using a Gerchberg-Saxton procedure.    -   3. Calculate the reconstruction intensity I of the hologram set,        oversampled to P×P pixels. Sum the intensities of the        reconstructions of each of the N holograms and divide the final        intensity by N. (An example of reconstruction of an image from        hologram data is described above, as part of the ADOSPR        procedure).    -   4. Calculate the intensity image F to display on the imaging        SLM. Set each pixel of F to the corresponding pixel of the        target image intensity T². Divide each pixel in F by the        corresponding pixel intensity in I. Let m be the maximum value        in the new field F. Then multiply each pixel in F by D/m.        Finally, set every pixel greater than 1 in F to 1.    -   5. The relative laser power K used to display this frame is        given by m/D.

The image can now be projected by displaying F on the imaging SLM, whilesequentially displaying the N hologram subframes on the hologram SLM.For a 30 Hz full-colour video input an N=16 holograms per colour plane,rates of 1.44 kHz and 90 Hz are required on the hologram and imaging SLMrespectively. For the avoidance of colour break-up, it can be preferableto increase, for example double these rates (2.88 kHz/180 Hz), withframes being duplicated as desired. FIG. 7 a shows, schematically, theeffect of the diffraction efficiency boost parameter, D.

FIG. 7 b shows a first example implementation of a holographicprojection system controller 202, including a digital signal processor210 operating under control of processor control code (which may beprovided on a storage medium such as Flash memory) to implement aprocedure as described above. FIG. 7 c shows an alternativeimplementation of an embodiment of the holographic projection systemcontroller 202, in dedicated hardware. In FIG. 7 c the image data isprovided to an input buffer 212 and thence to a hardware processingmodule 214 which comprises hardware to implement: a hologram targetgenerator, a super-resolution ADOSPR module (as described above), aholographic image reconstruction module, an intensity imagedetermination module to determine data to display on the imaging SLM,and preferably a relative laser power determination module. The hardwareprocessing module is coupled to working memory 216 and provides outputdata to an output buffer 218, which provides data outputs to drive theSLMs and to control the laser powers. The input and output buffers,hardware processing module and working memory operate under control of atiming and control block 220.

Simulated Results

The procedure described above was used to form hologram/image pairs fora Mustang image, size 640×640 pixels (see FIG. 1, left) and thesimulated reconstructions were calculated. First-order diffractionefficiency, RMS reconstruction error, and peak reconstruction error weremeasured. Diffraction efficiencies are given relative to a theoreticalcontinuous-phase holographic projection architecture of the type shownin FIG. 5 a.

Example SLM1 replay field (incident on SLM2 surface) and SLM2 data

-   -   8 phase levels    -   M=32×32-pixel SLM    -   N=16 subframes    -   D=1

FIG. 8 shows, on the left, a replay field I formed by 16 hologramsubframes displayed on a phase SLM—an ˜32×32 pixel SLM; the replay fieldis the Fourier transform of SLM1. FIG. 8, right, shows the correspondinghigh-frequency image to display on intensity modulating SLM2. Theproduct of the intensities of I and F gives the original Mustang imageintensity (FIG. 1) with zero reconstruction error, and a diffractionefficiency of 0.333.

Effect of SLM phase levels on performance

-   -   M=32×32-pixel SLM    -   N=16 subframes    -   D=1

SLM phase Diffraction RMS reconstruction Peak reconstruction levelsefficiency error error 2 0.161 0 0 3 0.263 0 0 4 0.268 0 0 8 0.333 0 016 0.294 0 0 32 0.290 0 0 Continuous 0.289 0 0

Effect of SLM resolution on performance

-   -   8 phase levels    -   N=16 subframes    -   D=1

Diffraction RMS reconstruction Peak reconstruction SLM size M efficiencyerror error 16 × 16 0.259 0 0 32 × 32 0.333 0 0 40 × 40 0.342 0 0 64 ×64 0.290 0 0 80 × 80 0.310 0 0 160 × 160 0.280 0 0

Effect of D on performance

-   -   8 phase levels    -   N=16 subframes    -   M=32×32-pixel SLM

DE boost Diffraction RMS reconstruction Peak reconstruction factor Defficiency error error 1.0 0.333 0 0 1.2 0.400 0.0009 0.087 1.4 0.4650.0048 0.155 1.6 0.526 0.0105 0.209 1.8 0.558 0.0224 0.255 2.0 0.5690.0391 0.293 3.0 0.599 0.1054 0.423 5.0 0.633 0.1759 0.553

Effect of subframe count Non performance (D=1)

-   -   8 phase levels    -   M=32×32-pixel SLM    -   D=1

Subframe Diffraction RMS reconstruction Peak reconstruction count Nefficiency error error 2 0.029 0 0 4 0.214 0 0 8 0.269 0 0 16 0.333 0 032 0.335 0 0

Effect of subframe count Non performance (D=1.6)

-   -   8 phase levels    -   M=32×32-pixel SLM    -   D=1.6

Subframe Diffraction RMS reconstruction Peak reconstruction count Nefficiency error error 2 0.046 0.0001 0.033 4 0.342 0.0032 0.166 8 0.4290.0053 0.209 16 0.526 0.0105 0.209 32 0.528 0.0091 0.159

The visual effect on image quality of varying D

-   -   8 phase levels    -   N=16 subframes    -   M=32×32-pixel SLM

The visual effect is shown in FIGS. 9 a to 9 f, for D=1.0, 1.3, 1.5,2.0, 4.0, and 8.0 respectively. It can be seen that the image qualitydeteriorates at D=2, whereas D=1.5 produces a high quality image withgood diffraction efficiency. The tables above suggest that, depending onthe application, a value of diffraction efficiency scaling parametervalue D in the range D=1.4 to 1.8, for example D=1.5 to 1.6 mayrepresent a good image quality-diffraction efficiency balance.

Applications for the above described systems include, but are notlimited to, the following: control room displays; data projection;mobile phones; PDAs; laptops; digital cameras; digital video cameras;games consoles; in-car cinema; navigation systems (in-car or personale.g. wristwatch GPS); head-up and helmet-mounted displays forautomobiles and aviation; watches; personal media players (for examplean MP3 player or personal video player); dashboard mounted displays;laser light show boxes; personal video projectors (a “video iPod®”concept); advertising and signage systems; computers (includingdesktops); remote control units; architectural fixtures incorporating aholographic image display system; and, more generally, any device whereit is desirable to share pictures and/or for more than one person atonce to view an image.

No doubt many other effective alternatives will occur to the skilledperson. It will be understood that the invention is not limited to thedescribed embodiments and encompasses modifications apparent to thoseskilled in the art lying within the spirit and scope of the claimsappended hereto.

What is claimed is:
 1. A method of displaying an image holographically,the method comprising: inputting display image data defining said imagefor display; processing said image data to determine first image datarepresenting a first spatial frequency portion of said image data andsecond image data representing a second spatial frequency portion ofsaid image data, wherein said second spatial frequency is higher thansaid first spatial frequency; displaying a hologram of said first imagedata on a spatial light modulator (SLM) to form a holographicallygenerated intermediate real image; and modulating said intermediate realimage using said second image data to display said image; wherein saiddisplaying of said hologram comprises modulating light from a laserlight source using said spatial light modulator, wherein said modulatingof said intermediate real image comprises intensity modulating saidintermediate real image using a second spatial light modulator, whereinsaid modulating comprises determining from said second image datamodulation data for driving said second spatial light modulator, andwherein said determining of said modulation data comprises adjustingsaid modulation data to increase a proportion of light transmitted bysaid intensity modulating.
 2. A method of displaying an imageholographically as claimed in claim 1 wherein said adjusting comprisesscaling said second image data to increase pixel values of said secondimage data.
 3. A method of displaying an image holographically asclaimed in claim 2 further comprising controlling an intensity of saidlight from said laser light source to compensate for said scaling.
 4. Amethod of displaying an image holographically; the method comprisinginputting display image data defining said image for display; processingsaid image data to determine first image data representing a firstspatial frequency portion of said image data and second image datarepresenting a second spatial frequency portion of said image data,wherein said second spatial frequency is higher than said first spatialfrequency; displaying a hologram of said first image data on a spatiallight modulator (SLM) to form a holographically generated intermediatereal image; and modulating said intermediate real image using saidsecond image data and projecting light from said modulated intermediatereal image to display said image, said projecting including forming asecond intermediate real image onto a diffuser.
 5. A method ofdisplaying an image holographically, the method comprising: inputtingdisplay image data defining said image for display, processing saidimage data to determine first image data representing a first spatialfrequency portion of said image data and second image data representinga second spatial frequency portion of said image data, wherein saidsecond spatial frequency is higher than said first spatial frequency;displaying a hologram of said first image data on a spatial lightmodulator (SLM) to form a holographically generated intermediate realimage; and modulating said intermediate real image using said secondimage data to display said image; wherein said displaying of saidhologram of said first image data comprises generating and displaying aplurality of temporal holographic subframes, wherein each of saidsubframes after a first said subframe includes noise at least partiallycompensating for noise in one or more previous said subframes.